Skip to content

Enable ipv6 on AWS#1109

Merged
hughcapet merged 1 commit intotriggerfrom
fix-ipv6-support
Apr 25, 2025
Merged

Enable ipv6 on AWS#1109
hughcapet merged 1 commit intotriggerfrom
fix-ipv6-support

Conversation

@hughcapet
Copy link
Copy Markdown
Member

various fixes to run Spilo on ipv6 k8s setup

@hughcapet hughcapet added the bugfix Bug fixes and patches, e.g. fixing of a production issue that is affecting the customer experience. label Apr 17, 2025

# Only small subset of environment variables is allowed. We don't want accidentally disclose sensitive information
for E in $(printenv -0 | tr '\n' ' ' | sed 's/\x00/\n/g' | grep -vE '^(KUBERNETES_(SERVICE|PORT|ROLE)[_=]|((POD_(IP|NAMESPACE))|HOSTNAME|PATH|PGHOME|LC_ALL|ENABLE_PG_MON)=)' | sed 's/=.*//g'); do
for E in $(printenv -0 | tr '\n' ' ' | sed 's/\x00/\n/g' | grep -vE '^((AWS_EC2_METADATA_SERVICE_ENDPOINT|KUBERNETES_(SERVICE|PORT|ROLE))[_=]|((POD_(IP|NAMESPACE))|HOSTNAME|PATH|PGHOME|LC_ALL|ENABLE_PG_MON)=)' | sed 's/=.*//g'); do
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jFYI: We inject two envars in IPv6 clusters to make the AWS SDKs do the right thing: https://github.com/zalando-incubator/kubernetes-on-aws/blob/f2a56bb4d26eb568541e2c64edfc63f8d5f166a0/cluster/manifests/02-admission-control/config.yaml#L49-L50

Not sure if AWS_EC2_METADATA_SERVICE_ENDPOINT_MODE is needed here, though.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I see that it likely is not relevant based on your wal-e changes in the ipv6-imds branch 👍 )

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

our backup tool uses AWS SDK which uses ipv4 imds endpoint by default and fails when env is not there. so here we prevent env cleanup on the container run

Copy link
Copy Markdown
Member Author

@hughcapet hughcapet Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(backup tool -- wal-g I mean, with wal-e (another backup tool), it unfortunately requires manual code intervention:) )
but both need this env, right

@hughcapet hughcapet marked this pull request as ready for review April 24, 2025 14:39
@mikkeloscar
Copy link
Copy Markdown
Member

👍

1 similar comment
@hughcapet
Copy link
Copy Markdown
Member Author

👍

@hughcapet hughcapet merged commit ce8bb9e into trigger Apr 25, 2025
6 of 7 checks passed
@hughcapet hughcapet deleted the fix-ipv6-support branch April 25, 2025 09:39
hughcapet added a commit that referenced this pull request Aug 6, 2025
- wal-g 3.0.5 (not 3.0.7, see: #1133)
- Support deleting old backups regardless of the current backup tool - #1133, #1150
- Make Spilo ipv6 compatible (on AWS) - #1108, #1109, #1112, #1113, #1117, #1123
- remove spilo_cmd - #1147
- add roaringbitmap and pgfaceting extensions - #1101, #1107
- add KUBERNETES_BOOTSTRAP_LABELS env for the upcoming Patroni feature - #1093, #1097
- fix LOG_S3_TAGS parsing logic - #1093
pando85 pushed a commit to pando85/spilo that referenced this pull request Aug 18, 2025
- wal-g 3.0.5 (not 3.0.7, see: zalando#1133)
- Support deleting old backups regardless of the current backup tool - zalando#1133, zalando#1150
- Make Spilo ipv6 compatible (on AWS) - zalando#1108, zalando#1109, zalando#1112, zalando#1113, zalando#1117, zalando#1123
- remove spilo_cmd - zalando#1147
- add roaringbitmap and pgfaceting extensions - zalando#1101, zalando#1107
- add KUBERNETES_BOOTSTRAP_LABELS env for the upcoming Patroni feature - zalando#1093, zalando#1097
- fix LOG_S3_TAGS parsing logic - zalando#1093
pando85 pushed a commit to pando85/spilo that referenced this pull request Aug 18, 2025
- wal-g 3.0.5 (not 3.0.7, see: zalando#1133)
- Support deleting old backups regardless of the current backup tool - zalando#1133, zalando#1150
- Make Spilo ipv6 compatible (on AWS) - zalando#1108, zalando#1109, zalando#1112, zalando#1113, zalando#1117, zalando#1123
- remove spilo_cmd - zalando#1147
- add roaringbitmap and pgfaceting extensions - zalando#1101, zalando#1107
- add KUBERNETES_BOOTSTRAP_LABELS env for the upcoming Patroni feature - zalando#1093, zalando#1097
- fix LOG_S3_TAGS parsing logic - zalando#1093
ldming added a commit to apecloud-inc/spilo that referenced this pull request Apr 5, 2026
* fix: wrong regex for wal retention (zalando#1026) (zalando#1058)

Co-authored-by: Rob Nickmans <nickmansrob@gmail.com>

* Add pg17 and remove pg12 in trigger (zalando#1059)

* Update Patroni to version 3.3.5 (zalando#1068)

* Pass TIMESCALEDB argument as environment variable (zalando#1069)

* sync trigger branch (zalando#1076)

* Patroni 4.0
* timescaledb 2.18.0

* Adjust timescaledb to 2.18.0 in delivery (zalando#1078)

* Sync trigger 03-05 (zalando#1080)

- Fix timescaledb installation logic
- Make leader tag value for native AWS configurable
- Retain multiple minor versions of timescaledb

---------
Co-authored-by: idanovinda <idanovinda@gmail.com>

* Properly handle Patroni bootstrap_labels config (zalando#1093)

Additionally fix LOG_S3_TAGS parsing logic

* Set proper default KUBERNETES_BOOTSTRAP_LABELS (zalando#1097)

* Add pg_roaringbitmap and pgfaceting (zalando#1101)

* Add roaringbitmap to extwlist (zalando#1107)

* Correctly format URLs when address is IPv6 (zalando#1108)

Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>

* Enable ipv6 on AWS (zalando#1109)

- keep AWS_EC2_METADATA_SERVICE* envs to enable wal-g
- use custom wal-e

* Move AWS_EC2_METADATA_SERVICE_ENDPOINT* env vars to env files (zalando#1112)

* Write imds env files for all prefixes (zalando#1113)

* Add trailing slash to AWS_EC2_METADATA_SERVICE_ENDPOINT (zalando#1117)

To support older boto3 versions

* Fix AWS_EC2_METADATA_SERVICE_ENDPOINT* env handling (zalando#1123)

* Write AWS_EC2_METADATA_SERVICE_ENDPOINT* env files only when actually needed

* Add pgvector to extension whitelist to allow non-superuser creation (zalando#1127)

* Updating wal-g version v3.0.5->v3.0.7 (zalando#1134)

* fix download url and use ubuntu 22.04 for wal-g (zalando#1135)

* Support deleting old backups regardless of the current tool (zalando#1133)

* fix wal-g download url and use ubuntu 20.04 (zalando#1141)

* Enable build image for arm64 in pull request (zalando#1144)

* remove spilo_cmd directory (zalando#1147)

* fix sorting (zalando#1150)

* pg_profile 4.10 and Patroni 4.0.6 (zalando#1151)

* Update postgis to version 3.6 (zalando#1153)

* Remove WAL-E and make WAL-G the default backup tool (zalando#1143)

* pg_basebackup server compression (zalando#1155)

* Allow admin selecting from hypopg views (zalando#1162)

* Add PG18, remove PG13 (zalando#1156)

- PG18 support
- Fix extension's build caching problem, which causes wrong pgversion symbols in extension's *.so
- Improve logic for timescaledb minor versions cleanup
- Build bg_mon from source

Not included for PG18:
- pglogical-ticker
- pgl-ddl-deploy

---------

Signed-off-by: Mikkel Oscar Lyderik Larsen <mikkel.larsen@zalando.de>
Co-authored-by: Polina Bungina <27892524+hughcapet@users.noreply.github.com>
Co-authored-by: Rob Nickmans <nickmansrob@gmail.com>
Co-authored-by: Ida Novindasari <idanovinda@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Bug fixes and patches, e.g. fixing of a production issue that is affecting the customer experience.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants